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FOREWORD 


This  research  and  development  was  conducted  in  support  of  Project  ZF57-525-001- 
022-03.06  (Improving  Human-Computer  Interaction  for  Command  and  Control  Systems) 
under  the  sponsorship  of  the  Naval  Sea  Systems  Command.  The  objective  of  this  project 
is  to  enhance  the  effectiveness  of  command  and  control  systems  through  improved  design 
of  the  human-computer  interface.  In  particular,  the  project  is  designed  to  examine  the 
impact  of  information  overload  on  threat  analysis  performance  in  anti-air  warfare  and  to 
recommend  procedures  for  reducing  the  impact  of  such  overload  on  operational  readiness. 

This  report  documents  the  initial  effort  toward  quantifying  limits  in  human  informa¬ 
tion  processing  that  are  associated  with  critical  command  and  control  operations.  The 
research  vehicle  reported  here  will  provide  the  basis  for  subsequent  investigations  of 
human  decision  and  information  processing  behavior. 

Appreciation  is  expressed  to  EMC  Jones  of  the  Naval  Training  Center,  San  Diego,  and 
to  Dr.  Carl  Englund  of  the  Naval  Health  Research  Center,  San  Diego,  for  their  assistance 
in  providing  research  subjects. 


JAMES  F.  KELLY,  JR. 
Commanding  Officer 


JAMES  J.  REGAN 
Technical  Director 
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SUMMARY 


Problem 


The  analysis  of  threat  is  a  critical  function  in  many  combat  systems,  especially  under 
the  conditions  of  high  information  load  that  typify  anti-air  warfare  (AAW)  operations.  To 
reduce  the  effect  of  information  overload  on  AAW  operator  performance,  it  is  necessary 
to  understand  more  precisely  the  relationship  between  task  load  and  decision  per¬ 
formance.  Proper  investigation  of  this  relationship  requires  the  development  of  a 
research  vehicle  that  embodies  major  features  of  the  AAW  threat  analysis  problem  and 
permits  detailed  quantitative  measurement  of  human  performance. 

Objective 

The  objective  of  this  effort  was  to  develop  and  test  a  system  for  investigating  human 
performance  in  the  conduct  of  AAW  threat  analysis.  The  system  had  to  be  flexible,  be 
portable,  require  minimal  training,  induce  a  high  motivation  to  perform,  and  provide 
detailed  measurement  of  human  performance. 

Approach 

A  simulation  of  the  AAW  threat  analysis  problem  was  developed  and  embedded  in  an 
interactive  "air  defense  game"  driven  by  a  Tektronix  microcomputer.  The  game 

simulated  hostile  air  targets  approaching  the  player's  ship  at  one  of  three  speeds.  The 
entire  scenario  was  displayed  on  the  computer's  CRT  screen.  Task  difficulty  was 
manipulated  under  computer  control  by  varying  the  number  of  targets  and  their  arrival 
times.  The  player  defended  the  ship  by  launching  "missiles"  at  the  incoming  targets.  The 
ship's  detection  range  exceeded  the  range  of  its  missiles,  however,  so  that  a  launch-time 
decision  was  necessary  for  each  target.  The  goal  for  the  player  was  to  kill  all  threats  at 
maximum  range,  but  missiles  launched  too  early  would  fall  short  of  the  target  and  be 
ineffective. 

Following  each  engagement,  the  computer  program  gave  performance  feedback  to 
the  player  and  stored  all  relevant  data  for  subsequent  review  by  the  experimenter.  A  data 
analysis  program  was  developed  to  provide  details  of  player  performance. 

Seventeen  Navy  enlisted  men  served  in  an  initial  experiment  to  evaluate  the  system's 
utility  for  human  performance  research.  Task  variables  were  the  number  of  targets  in 
each  engagement,  the  pace  or  tempo  of  operations,  and  target  speed.  The  effects  of  two 
different  training  sequences  and  of  a  concurrent  auditory  monitoring  task  were  also 
investigated. 

Findings 


1.  The  feasibility  of  the  software  concept  and  its  implementation  were  confirmed 
by  the  preliminary  experiment. 

2.  On-line  extraction  of  performance  data  permitted  researchers  to  compare  the 
performance  of  the  players  with  that  of  a  mathematically  ideal  information  processor. 

3.  Test  subjects  became  proficient  after  about  3  hours  of  practice.  Training 
manipulations  had  little  effect  on  the  course  of  skill  acquisition. 
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4.  The  effects  of  task  load  were  evidenced  by  a  decline  in  performance  as  the 
number  of  targets  and  the  pace  of  operations  were  increased.  Performance  was  also 
impaired  by  the  introduction  of  a  concurrent  auditory  monitoring  task. 

5.  The  air  defense  game  was  challenging  and  induced  a  high  level  of  motivation. 
Test  subjects  sustained  their  attention  to  the  task  for  3  to  4  hours  at  a  sitting. 

Conclusions 


The  air  defense  game  offers  a  rich  analogue  of  AAW  threat  operations.  It  is  readily 
learned,  motivating  to  perform,  and  provides  an  effective  vehicle  for  human  performance 
research. 

Recommendations 


1.  The  air  defense  game  should  be  exploited  as  a  tool  in  future  research  on  the 
threat  analysis  problem. 

2.  The  air  defense  game  should  be  used  to  determine  limitations  in  human 
information  processing  and  to  find  out  how  threat  analysis  strategies  change  as  a  function 
of  task  load. 

3.  Performance  in  the  air  defense  game  should  be  used  as  a  dependent  measure  in 
other  military  research  applications  (e.g.,  the  effect  of  sleep  loss  or  extended  effort). 
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INTRODUCTION 


Problem 


The  analysis  of  threat  is  a  critical  function  in  many  combat  systems,  especially  under 
the  conditions  of  high  information  load  that  typify  anti-air  warfare  (AAW)  operations.  In 
order  to  reduce  the  effect  of  information  overload  on  AAW  operator  performance,  it  is 
necessary  to  understand  more  precisely  the  relationship  between  task  load  and  decision 
performance.  Proper  investigation  of  this  relationship  requires  the  development  of  a 
research  vehicle  that  embodies  major  features  of  the  AAW  threat  analysis  problem  and 
permits  detailed  quantitative  measurement  of  human  performance. 

Objective 

The  objective  of  this  work  was  to  develop  and  test  a  system  for  investigating  human 
performance  that  would: 

•  Exercise  the  complex  cognitive  processes  encountered  in  AAW  threat  analysis 
operations. 

•  Require  minimal  training. 

•  Make  minimal  psychotnotor  demands. 

•  Enable  detailed  performance  measurements. 

•  Have  a  military  character  and  be  inherently  motivating  to  perform. 

•  Use  a  single  stand-alone  microcomputer. 

•  Be  flexible  enough  to  enable  the  use  of  varied  scenarios. 

•  Permit  ready  evaluation  of  its  operation. 

Background 

As  the  naval  tactical  environment  becomes  more  complex  and  fast-paced,  the  need 
for  timely  and  effective  decisions  imposes  an  increasingly  heavy  burden  on  command. 
Human  performance  and  the  design  of  the  human-computer  interface  become  critical. 
Thus,  it  is  imperative  that  combat  systems  be  designed  to  minimize  potential  performance 
deficits  under  conditions  of  heavy  information  load.  Proper  guidelines  for  design  should 
include  quantitative  human  performance  data,  as  has  been  emphasized  by  representatives 
of  NAVELEX,  fleet  sites,  and  the  Chief  of  Naval  Operations  (Command,  Control,  and 
Communications  Programs  Office  (OP -94)).  The  Navy  Personnel  Research  and  Develop¬ 
ment  Center  (NAVPERSRANDCEN)  has  an  ongoing  research  effort  that  seeks  to  quantify 
human  performance  limits  in  dealing  with  AAW  threats. 

The  problem  of  information  overload  is  especially  acute  at  the  level  of  the  individual 
ship,  where  the  Cofnbat  Direction  Center  staff  must  integrate  data  from  several  sources 
to  detect,  classify,  and  monitor  contacts.  Staff  personnel  must  attend  concurrently  to 
multiple  contacts  and  make  accurate  and  timely  decisions  under  stress  (Combat  Direction 
Systems  Department  Organizational  Manual,  USS  AMERICA  (CV  66),  1978;  Cullison,  1979; 
Halnon,  1979). 


The  analysis  of  threat  in  tactical  operations  is  largely  a  covert,  cognitive  activity 
about  which  little  is  known.  In  general  terms,  decision  makers  must  recognize  contacts  as 
potential  threats,  set  priorities,  and  initiate  appropriate  actions  to  neutralize  contacts 
determined  to  be  hostile.  A  Ship  Weapons  Coordinator  (SWC),  for  example,  must  respond 
to  messages  and  alerts  from  the  staff,  monitor  the  status  of  potential  threats,  assign  the 
appropriate  weapons,  and  carry  out  directives  from  his  superiors.  This  sequence  works 
reasonably  well  in  routine  situations,  but  problems  occur  when  track  loads  become  heavy, 
as  they  often  do  in  AAW  operations.  Here  the  rate  of  information  flow  increases 
dramatically  and  performance  begins  to  degrade.  The  SWC's  job  is  especially  vulnerable 
to  information  overload. 

A  recurring  problem  in  the  study  of  information  overload  has  been  the  selection  of  a 
measure  of  cognitive  performance  that  is  sensitive  to  variations  in  task  load.  The  more 
successful  studies,  such  as  those  by  North  and  Gopher  (1976)  and  Wickens  and  Gopher 
(1977),  have  addressed  the  aircraft  environment.  Typically,  a  compensatory  tracking  task 
has  been  used  to  simulate  the  psychomotor  demands  of  flying.  The  tracking  task  was 
performed  in  conjunction  with  other  tasks,  such  as  map  reading,  and  tracking  error  was 
then  taken  as  an  indicator  of  mental  workload.  Harris,  North,  and  Owens  (1978)  have 
described  an  experiment  controller  system  that  employs  this  paradigm.  The  specifics  of 
this  dual-task  paradigm  cannot  be  applied  directly  to  AAW,  however,  because  AAW 
problems  place  little  or  no  reliance  on  psychomotor  skills  like  tracking.  Instead,  AAW 
subsystems  require  sustained  attention  to  complex  information  processing  tasks  in  which 
threat  analysis  has  high  priority.  Therefore,  at  the  start  of  this  effort,  a  research  tool 
was  needed  that  would  incorporate  the  cognitive  features  of  AAW'  threat  analysis,  provide 
close  experimental  control  of  task  events,  and  yield  detailed  quantitative  measurements 
of  human  performance. 


AIR  DEFENSE  GAME 

A  laboratory  simulation  of  the  AAW  threat  analysis  problem  was  developed  and 
embedded  in  an  "air  defense  game"  driven  by  a  Tektronix  ^051  microcomputer.  This 
approach  captured  the  important  human  information  processing  demands  of  AAW  threat 
analysis  by  requiring  sustained  attention  to  a  complex  "hostile"  environment  that  unfolded 
in  real  time  and  responded  appropriately  to  the  actions  of  the  player.  At  the  same  time, 
the  simulation  scheme  met  the  researchers'  requirements  for  experimental  control  and  for 
quantitative  measurements  of  human  performance.  The  computer  is  shown  in  Figure  1. 

Scenario 


The  computer  supervises  an  AAW  scenario  in  which  a  player  defends  a  ship  by 
launching  "missiles"  at  "hostile  targets"  that  appear  on  the  computer's  CRT  screen. 
Instructions  given  to  the  player  appear  in  Figure  2.  The  goal  is  to  kill  all  targets  at 
maximum  range,  but  the  ship's  detection  range  exceeds  its  weapons  range  so  that  a 
launch-time  decision  is  necessary  for  each  target.  Missiles  launched  too  early  fall  short 
of  their  targets  (i.e.,  splash).  It  is  possible  to  fire  again  after  a  splash,  but  only  one 
missile  is  allowed  to  be  in  flight  on  a  given  track  at  one  time.  Duplicate  (inflight) 
launches  result  in  penalties.  Penalties  are  also  incurred,  of  course,  when  targets  hit  the 
ship. 
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IMM 


■igure  1.  Computer  displaying  the  air  delense  game  in 


AIR  DEFENSE  GAME  INSTRUCTIONS 

YOUR  SHIP  IS  UNDER  ATTACK  BY  INCOMING  MISSILES  WITH 
VARYING  SPEEDS  AND  LAUNCH  TIMES.  YOUR  TASK  IS  TO  MONITOR 
THE  RADAR  DISPLAY  AND  DESTROY  THEM,  THE  PRIORITIES 
OF  THE  ENGAGEMENT  ARE: 

<1>  AVOID  BEING  HIT. 

<2/  AVOID  SPLASHING  YOUR  OWN  MISSILES  BY  LAUNCHING  TOO 
EARLY... YOUR  WEAPONS  RANGE  IS  20  MILES.  WHICH 
IS  THE  INNER  CIRCLE  OP  THE  RADAR  DISPLAY. 

(3^  DESTROY  INCOMING  MISSILES  AS  SOON  AS  POSSIBLE 
AFTER  THEY  ENTER  YOUR  WEAPONS  RANGE. 

(4)  AVOID  LAUNCHING  A  MISSILE  IF  YOU  ALREADY  HAUE  ONE 
IH  FLIGHT  ON  THE  SAfiE  TARGET, 

YOUR  SKILL  RATING  cO-lOO)  HILL  INCLUDE  A  12-POINT 
PENALTY  FOR  EACH  HIT  SUSTAINED  AND  A  2-POINT  PENALTY  FOR 
EACH  INFLIGHT  LAUNCH.  THE  MAXIMUM  KILL  RANGE  IS  20  MILES, 

FIVE  POINTS  APE  DEDUCTED  FOR  EACH  MILE  THAT  YOUR  AVERAGE 
KILL  RANGE  IS  UNDER  20. 

TO  LAUNCH  A  MISSILE,  USE  THE  TEH  WHITE  KEYS  AT  THE  UPPER 
LEFT  OF  THE  KEYBOARD.  ENTER  THE  TWO-DIGIT  TRACK  NUMBER 
OF  THE  TARGET... A  READOUT  WILL  THEN  INFORM  YOU  OF  A  SUCCESSFUL 
LAUNCH— OR  AN  ERROR.  AN  ERROR  OCCURS  IF  YOU  KEY  A  HUMBER 
IHCCPRECTi-Y  OR  IF  YOU  LAUNCH  A  MISSILE  UNNECESSARILY. 

GOOD  LUCK _ THE  FATE  OF  YOUR  SHIP  LIES  IH  YOUR  HANDS! 


Figure  2,  CRT  display  of  instructions  for  air  defense  game. 
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Game  Activity  i 

Figure  3  shows  a  sample  snapshot  of  the  game  in  progress.  The  display  simulates  a 
radar  screen  in  which  the  ship  is  the  "+"  sign  at  the  center,  the  inner  circle  is  the  weapons 
range  (20  miles),  and  the  outer  circle  is  the  radar's  detection  range  (46  miles).  (The  radius 
of  the  outer  circle  is  13.5  centimeters.)  Targets  appear  as  dots  (blips)  on  the  display  and 
each  is  randomly  assigned  a  unique  track  number  (TN),  01  through  99,  that  appears  at  the 
outer  circle  as  soon  as  the  target  enters  the  display.  All  targets  head  directly  toward  the 
ship  at  randomly  selected  bearings. 
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The  computer,  simulating  a  radar  sweep,  updates  the  display  every  11.7  seconds,  and 
at  each  update  another  blip  is  displayed  on  all  target  paths.  The  distance  between  blips, 
then,  is  a  cue  to  the  relative  speed  of  the  targets.  In  Figure  3,  TN  24  is  fast,  TN  30  is 
medium,  and  TN  60  is  slow;  their  respective  speeds  are  5,  3,  and  1.6  miles  per  update. 
The  sweep  time  varies  from  0.6  to  0.8  seconds  and  the  time  between  sweeps  is  available 
for  action  by  the  player  and  processing  by  the  computer. 

The  player  launches  missiles  by  pressing  function  keys  at  the  upper  left  of  the 
computer's  keyboard.  Each  keypress  codes  one  digit  of  the  required  2-digit  TN.  Feedback 
for  a  good  launch  (i.e.,  at  a  valid  track)  is  indicated  at  the  upper  left  of  the  display  by 
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flashing  the  keyed  TN  next  to  the  letter  "L"  (for  "launched").  An  illegal  launch  attempt 
(i.e.,  for  a  nonexistent  track  or  if  a  missile  is  already  in  flight  on  the  track)  is  indicated  by 
flashing  the  keyed  TN  next  to  the  letter  "E"  (for  "error"). 


When  the  player  launches  a  missile,  a  dash  (-)  is  immediately  displayed  to  represent 
its  path,  and  another  dash  is  added  at  each  update  thereafter.  Missiles,  like  fast  targets, 
travel  5  miles  per  update.  When  the  missile  reaches  its  intended  target,  a  kill  occurs,  a 
bell  sounds,  and  the  appropriate  radius  is  filled  by  a  solid  line  on  the  display.  If  an 
incoming  target  reaches  the  ship,  a  hit  occurs,  a  longer  bell  sounds,  and  the  TN  of  the 
target  is  blocked  out. 

In  the  example  shown  in  Figure  3,  the  player  has  destroyed  TN  97  and  has  sustained  a 
hit  from  the  target  on  the  track  at  4  o’clock.  A  missile  has  just  been  launched  at  TN  84, 
and  missiles  were  launched  against  TNs  24  and  27  two  sweeps  earlier.  A  splash  has 
occurred  on  TN  25;  that  missile  reached  weapons  range  too  early  to  kill  its  assigned 
target. 

Player  Feedback 

A  single  play  of  the  game  continues  until  each  target  in  the  scenario  has  either  been 
killed  or  has  hit  the  ship.  The  screen  is  then  erased  and  the  player  is  given  feedback  in 
the  form  of  an  air  defense  summary  (see  Figure  4).  The  "average  range  for  kills"  given 
in  the  summary  is  the  mean  distance  from  the  ship  to  the  points  where  targets  were 
destroyed  by  missiles  (hits  are,  of  course,  excluded  from  this  computation).  An  overall 
"skill  rating"  (R)  is  provided  that  takes  each  of  the  engagement  priorities  into  account. 
The  rating  is  defined  as: 

R  =  100  (Average  kill  range/20)  -  12  (No.  of  hits)  -  2  (No.  of  inflights). 


AIR  DEFENSE  SUNHARY 

PLAYER  NO.  10 

HO.  OF  TARGETS  «  IS  TEMPO: 

intermediate 

SPEED  i  PILLS  * 

HITS 

FAST  5 

1 

HEDIUM  6 

0 

SLOH  e 

0 

TOTAL  KILLS  »  17  TOTAL  HITS 

=  I 

AUERACE  RANGE  FOP  KILLS  =•  16.42 

NO.  OF  MISSILES  LAUNCHED  »  18 

HO.  OF  MISSILES  SPLASHED  =  1 

NO.  OF  INFLIGHT  LAUNCHES  *  0 

(MAX.  *  20) 

SKILL  RATING  *  70  (MAX.  »  lOOT 

Dato  IS  beiTia  stored  on  tape... 

DO  YOU  WANT  TO  PLAY  AGAIN  (Y/N)' 

Figure  4.  Feedback  information  displayed  at  the 
end  of  each  engagement. 
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If  performance  is  perfect,  all  targets  will  be  killed  at  the  maximum  range  of  20 
miles,  there  will  be  no  hits  or  inflight  launches,  and  R  will  be  100.  While  incurred 
penalties  can  render  R  <  0,  R  =  0  was  the  minimum  value  displayed  as  feedback  to  the 
player. 

Task  Variables 


The  major  variables  affecting  the  conduct  of  the  air  defense  game  are  the  number  of 
targets,  the  tempo  of  operation,  and  target  speed. 

Number  of  Targets 

The  size  of  the  computer's  memory  limits  the  number  of  targets  presented  in  any 
engagement  to  72.  A  displayed  menu  makes  five  selections  available:  6  (for  practice 
only),  18,  36,  54,  or  72.  Equal  numbers  of  fast,  medium,  and  slow  targets  are  always 
assigned.  The  minimum  separation  between  tracks  is  4°. 

Tempo  of  Operation 

The  tempo  is  also  selected  from  a  menu  and  controls,  through  one  of  three 
randomization  routines,  the  rate  at  which  the  targets  appear.  In  the  low  tempo,  target 
entry  times  are  relatively  spread  out  and  prosecuting  the  threats  is  comparatively  easy. 
As  the  tempo  is  raised,  the  information  processing  demands  increase.  Entry  time 
patterns,  though  varying  with  tempo  and  from  engagement  to  engagement,  always  have 
the  following  characteristics: 

•  Slow  targets,  which  require  the  longest  time  to  reach  the  ship,  generally  enter 
early  in  the  engagement. 

•  Medium-speed  targets  enter  most  prominently  during  the  middle  of  the  engage¬ 
ment. 

•  Fast  targets  have  their  entries  distributed  over  the  entire  engagement,  except 
for  their  absence  in  the  first  several  updates. 

For  any  fixed  number  of  targets,  increasing  the  tempo  increasingly  crowds  the  times 
of  entry.  The  effect  is  to  shorten  the  engagement  and  yet  preserve  the  patterns  of  entry 
times  just  described.  This  can  be  seen  in  Figure  5,  which  depicts  the  entry-time 
distributions  of  18  targets  for  the  three  tempos.  Each  horizontal  axis  is  the  sequence  of 
display  updates,  and  the  letters  F,  M,  and  S  (six  of  each)  indicate  the  entry  of  fast, 
medium,  and  slow  targets.  Each  entry  is  stochastic  (i.e.,  each  target  is  equally  likely  to 
enter  at  its  indicated  update  U  or  at  U  +  1).  In  effect,  a  fair  coin  is  tossed  separately  for 
each  of  the  18  targets  to  determine  its  actual  entry  time  in  the  given  engagement. 

The  pattern  of  entry  times  given  in  Figure  5  does  not  imply  a  similar  pattern  for 
missile  launches.  Targets  entering  the  display  at  different  times  with  different  speeds 
may  very  well  enter  the  ship's  missile  range  at  the  same  time.  The  effect  of  each 
schedule  is  to  create  a  peak  demand  for  missile  launches  during  the  middle  of  each 
engagement. 
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For  any  engagement,  the  number  of  targets  and  the  tempo  combine  to  determine  the 
number  of  updates  to  the  entry  of  the  last  target,  the  mean  number  of  updates  between 
successive  target  arrivals,  and  the  approximate  duration  of  the  engagement.  The  actual 
duration  depends  in  part  upon  the  player's  proficiency  in  responding  to  those  tracks  that 
appear  near  the  end  of  the  game.  Engagement  characteristics  are  summarized  in  Table  1. 


Target  Speed 


As  previously  stated,  the  three  target  speeds  are  5.0,  3.0,  and  1.6  miles  per  update, 
with  initial  ranges  at  entry  being  45,  44,  and  44  miles  respectively.  If  the  player  fails  to 
fire,  a  fast  target  will  hit  the  ship  at  its  10th  update;  a  medium  target,  at  its  16th  update; 
and  a  slow  target,  at  its  28th  update. 


Performance  Measurement  and  Analysis 


A  major  advantage  of  on-line  computerized  control  is  that  details  of  the  player's 
behavior  can  be  sensed  and  stored  automatically.  Here,  each  missile  launch  by  the  player 
is  stored  in  memory  and  identified  with  its  track  number  and  the  display  update  on  which 
it  occurred.  The  parameters  and  schedules  of  the  targets  also  reside  in  memory.  These 
data  are  stored  on  magnetic  tape  at  the  end  of  each  game  so  that  the  engagement  can  be 
reconstructed  for  the  experimenter  and  analyzed  in  detail. 


A  separate  off-line  analysis  program  retrieves  the  data  from  tape  and  displays  an 
overall  performance  summary  identical  to  that  shown  in  Figure  4.  Additional  analyses 
provide  the  experimenter  with  more  detailed  summaries  of  the  engagement.  Fundamental 
to  these  analyses  is  the  notion  that  there  is  an  optimal  launch  time  that  can  be  used  as  a 
standard  for  player  performance. 
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Table  1 

Engagement  Characteristics  as  Determined  by 
the  Number  of  Targets  and  the  Tempo  of  Operation 


Number 

of 

Targets 

Tempo  of  Operation 

Low 

Intermediate 

High 

Number  of  Updates  to  Last  Target  Entry^ 

18 

30 

19 

14 

36 

60 

34 

24 

5k 

90 

49 

34 

72 

120 

64 

44 

Mean  Number  of  Updates  Between  Successive  Target  Arrivals 


IS 

1.70 

1 .06 

.76 

36 

1.68 

.94 

.  66 

54 

1.68 

.90 

.62 

72 

1.68 

.89 

.60 

Duration  of  Engagement  in  Minutes*^ 

18 

7.2 

5.0 

4.2 

36 

13.0 

8.0 

6.0 

54 

18.5 

10.8 

8.0 

72 

24.8 

14.2 

9.9 

Note:  Updates  occur  at  intervals  of  approximately  11.7  seconds.  The  precise  interval 
depends  upon  the  status  of  the  targets  and  the  actions  of  the  player. 

^The  randomization  procedure,  with  probability  =  '/2,  adds  one  to  the  tabled  value. 

^These  are  approximate  playing  times  for  a  single  engagement.  Actual  durations  vary 
somewhat  with  random  scheduling  effects  and  the  proficiency  of  the  player. 


Optimal  Launch  Time 

If  player  performance  is  optimal,  each  target  will  be  destroyed  at  precisely  the  20 
mile  weapons  range  circle.  For  a  fast  target  entering  the  display  at  update  E,  the  optimal 
launch  time  (L*)  is  at  update  E  +  2,  which  corresponds  to  the  display  of  its  third  blip.  For 
a  medium  target  L*  =  E  +  5,  its  sixth  blip.  Similarly,  for  slow  targets,  we  have 
L*  =  E  +  12. 
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As  one  performance  measure,  the  player's  actions  are  compared  with  the  optimal 
tactics  by  taking  the  difference  between  observed  launch  time  (L)  and  optimal  launch 
time  (L*)  to  yield  the  lag  =  L  -  L*.  The  ideal  player  will  have  zero  lags  for  all  targets, 
and  increasing  lags  generally  represent  poorer  performance  by  the  player.  If  a  target  hits 
the  ship,  we  arbitrarily  assign  a  lag  =  -  L*.  Firing  too  early  (a  splash)  would  also  result  in 
a  lag  <  0,  but  subsequent  behavior  by  the  player--either  a  kill  or  allowing  a  hit  to 
occur--determines  the  actual  lag  assigned  to  that  target. 

Target  Information  and  Weapons  Employment  Summary 

Figure  6  shows  an  example  of  a  target  information  and  weapons  summary.  For  each 
of  the  18  targets  in  this  engagement,  the  top  part  of  the  display  lists  the  target  speed  (F, 


M,  or  S)  and  the  following; 

•  Bearing  angle  in  degrees . ANG 

•  Track  number  (01  through  99) .  TN 

•  Entry  time  (update  number) .  E 

•  Optimal  launch  time  (update  number) .  L* 

•  Actual  launch  time  (update  number) .  L 


ANC 

TN 

E 

L* 

KL 

LAG 

IN 

SP 

0 

1 

F 

112 

46 
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0 
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24 

10 

12 

12 
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0 

0 

K 

3 
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10 
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18 

19 

1 

0 

0 

K 

4 

A 

27 

10 

12 

12 

0 

0 

0 

K 

5 

F 
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13 

14 

16 

IS 

2 

0 

0 

K 
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F 
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58 

20 

22 

22 
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0 

0 

K 

M 

36 

21 

4 

"9 

15 

6 

0 

0 

K 
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M 
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30 

8 

13 

15 

2 

0 

0 

K' 

(5 

M 
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53 

12 

17 

18 

1 
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0 

K 

10 

M 
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97 

5 

10 

10 

0 

0 

0 
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M 
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26 

10 

15 

18 

3 

0 

0 

K 
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41 

13 

18 

19 

1 

0 

0 

K 

13 

S 

12 

63 

1 

13 

16 

3 
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0 

K 

H 

s 

40 

25 

4 

16 

17 

1 

0 

1 

K 

15 
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60 

5 

17 

19 

2 
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0 
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14 
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2 
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55 
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18 

2 
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0 

K 
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c 

o 

D  LAG 
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TARGET 
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0 
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Figure  6.  Summary  of  target  information  and  weapons 
employment  for  an  18-target  engagement. 
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•  Lag  time  (L  -  L*)  .... 

•  Number  of  inflight  launches  .  IN 

•  Number  of  splashes .  SP 

•  Outcome  (K  =  target  killed,  H  =  hit  on  ship) .  O 


The  bottom  of  the  display  shows  the  mean  lags  and  standard  deviations  (SD  LAG)  for 
each  target  speed  and  for  all  targets  combined.  The  sample  size  (N)  for  these  statistics 
includes  only  those  targets  that  are  killed  by  the  player.  Finally,  the  number  of  inflight 
launches  (IN)  and  splashes  (SP)  are  totaled  for  each  target  speeid. 

Dynamic  Performance  Summary 

The  above  analyses  refer  to  individual  targets  or  to  averages  over  targets  of  a  given 
speed.  As  such,  they  do  not  capture  the  player's  performance  as  the  engagement  unfolds 
in  time.  To  obtain  this  kind  of  dynamic  profile,  the  analysis  program  summarizes  the 
situation  at  each  update  of  the  engagement.  Figure  7  shows  such  a  summary  for  an  18- 
target  engagement  concluded  in  25  updates.  The  summary  lists  the  following  data: 


•  Update  number .  Ij 

•  Number  of  active  targets .  ACT 

•  Number  of  targets  engaged  by  missiles .  LNG 

•  Cumulative  number  of  targets  killed .  K 

•  Cumulative  number  of  hits  on  ship .  H 

•  Number  of  targets  that  remain  scheduled  to  appear .  REM 

•  Number  of  unengaged  targets  within  firing  range .  UT 

•  Number  of  missed  opportunities  to  fire  at  eligible  targets .  MO 


The  UT  and  MO  scores  are  candidate  measures  that  seek  to  capture,  at  each  update, 
the  extent  to  which  the  player's  actions  are  falling  behind  optimal  performance.  The  UT 
measure  is  simply  the  number  of  eligible  targets  that  have  yet  to  be  fired  on  at  the 
current  update. 

Note  that  the  UT  measure  does  not  consider  the  number  of  prior  updates  on  which  a 
target  has  been  eligible.  The  alternative  measure  MO  includes  this  history.  It  is  derived 
as  follows:  For  the  current  update,  denote  the  UT  unengaged  targets  within  range  as 
^l’"'’^UT'  forget,  Tj,  has  been  in  this  state  for,  say  Xj  updates  including  the 

current  one.  The  measure  MO  is  then  the  sum  of  the  x.  for  i  =  1,...,UT.  Thus,  it  should  be 

seen  that  MO  is  the  number  of  missed  opportunities  to  fire  at  eligible  but  unengaged 
targets.  Equivalently,  for  any  snapshot  of  the  display  at  a  given  update,  MO  equals  the 
total  number  of  displayed  blips  within  firing  range  for  unengaged  targets. 
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Figure  7.  Dynamic  performance  summary. 


MO  and  UT,  then,  provide  related  measures  of  the  player's  efficiency  as  the 
engagement  unfolds  over  time.  It  should  be  clear  that  MO>UT  and  that,  for  an  ideal 
player  who  fires  at  the  optimal  launch  time,  MO  =  UT  =  0  for  all  updates.  Real  players, 
however,  cannot  process  all  targets  as  quickly  as  they  become  eligible. 

A  profile  of  MO  and  UT  is  displayed  by  the  analysis  program,  as  shown  in  Figure  8. 
For  the  UT  index  (the  "+"  signs),  the  first  target  came  into  firing  range  at  the  7th  update; 
the  player  did  not  launch  a  missile  at  that  time,  so  UT  =  1  for  update  7.  There  were  two 
eligible  but  unengaged  targets  on  the  updates  9  through  12,  etc.  As  illustrated  by  the  UT 
profile,  this  player  progressively,  and  typically,  fell  behind  optimal  performance  during 
the  higher-paced  middle  portion  of  the  engagement.  The  MO  index  (the  solid  line)  reveals 
a  similar  impairment  of  performance  during  the  period  of  peak  processing  demand.  The 
asterisk  in  Figure  8  indicates  that  a  hit  was  sustained  on  the  14th  update. 
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=  number  of  unengaged  targets  within  firing  range. 

=  number  of  missed  opportunities  to  fire  at  such  targets. 
=  first  target  enters  firing  range. 

=  hit(s)  sustained  at  this  update. 


Figure  8.  Two  performance  indices  as  functions  of  update. 


Hardware  Characteristics 


A  Tektronix  4051  microcomputer  controls  the  air  defense  simulation  and  the 
subsequent  analyses  of  player  performance.  It  has  32K  bytes  of  memory,  a  direct-view 
storage  tube  display  (19.1  cm  by  14.0  cm),  and  a  magnetic  tape  cartridge  drive  for  storage 
of  programs  and  data  files.  The  programming  language  is  Graphic  System  BASIC 
(Tektronix,  1976).  The  speed  of  graphic  and  arithmetic  operations  was  enhanced  by 
attaching  an  FP-51  Read-Only-Memory  Pack.^ 

The  keyboard  has  the  standard  typewriter  layout,  an  auxiliary  numeric  entry  keyset, 
and  10  user-definable  function  keys  at  the  upper  left.  These  latter  were  numbered  from 
0-9  and  were  used  to  enter  track  numbers  to  launch  missiles.  The  function  keys  were  the 
only  ones  used  during  engagements;  a  Plexiglas  sheet  covered  the  main  keyboard  to 
prevent  inadvertent  key  presses. 

Program  Flow 

The  program  listing  for  the  air  defense  game  appears  in  Appendix  A;  a  list  and 
description  of  variables  is  given  in  Appendix  B.  The  program  flow  for  the  game  is  shown 
in  Figure  9.  The  first  part  of  the  program  (START  to  node  B  in  the  flowchart)  provides 
instructions  to  the  player  and  sets  up  the  engagement  parameters.  The  major  section 
(nodes  B  to  G)  controls,  for  each  update,  the  scheduled  entry  and  display^  of  targets  and 
missiles,  tallies  the  player's  actions  and  other  events,  and  generally  supervises  the 
progress  of  the  scenario.  At  the  end  of  the  engagement,  the  last  section  of  the  main 
program  (Figure  9-c)  controls  the  feedback  display  and  the  storage  of  data  on  magnetic 
l^pe.  A  special  weapon-launching  routine  (Figure  9-d)  interrupts  the  main  program  to 
process  missile  firings  as  they  are  keyed  in  by  the  player.  From  the  player's  perspective, 
the  interrupt  software  causes  no  disruption  of  the  flow  of  the  game. 

In  the  off-line  data  analysis  program,  the  program  flow  is  straightforward:  the  user 
selects  the  file  to  be  analyzed  and  sequences  through  the  displayed  output.  The  program 
listing  is  in  Appendix  C;  the  variables  and  their  descriptions  are  in  Appendix  D. 

Limitations 


The  characteristics  of  the  hardware  and  software  impose  the  following  limitations; 

•  Memory  size  limits  the  number  of  targets  to  72. 

•  The  11.7  second  interval  between  updates  of  the  simulated  radar  display  cannot 
be  reduced  while  maintaining  a  constant  inter-update  interval  for  all  combinations  of  the 
task  variables.  If  desired,  the  interval  can  be  increased  by  the  straightforward  insertion 
of  a  dummy  routine. 


‘Micro  Works,  P.O.  Box  1110,  Del  Mar,  California  92014. 

^To  reduce  variability  in  display  timing,  on  every  update  the  program  has  an 
inventory  of  90  targets  that  are  spaced  4®  apart.  Only  the  active  ones  are  displayed;  all 
others  are  suppressed. 


Figure  9a.  Summary  flowchart  of  air  defense  game 


Figure  9b. 
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GAME 


Figure  9d, 


•  The  display  is  a  storage  tube  and  cannot  be  selectively  erased;  thus,  the  paths  of 
all  targets  and  missiles  must  remain  on  the  screen  for  the  duration  of  the  engagement. 
This  tends  to  clutter  the  display  when  the  number  of  targets  is  large. 

•  There  is  no  provision  for  other  than  straight-line  tracks.  No  change  of  target 
course  is  permitted. 

•  The  function  keys  must  be  pressed  and  released  quickly  to  avoid  input  errors. 
Practice  sessions  can  serve,  in  part,  to  familiarize  the  user  with  proper  striking  of  the 
keys. 

•  The  program  requires  the  FP-5I  accessory  pack  in  order  to  execute  selected 
graphical  and  mathematical  commands. 


AN  EXPERIMENT  TO  DEMONSTRATE  FEASIBILITY 

To  demonstrate  the  feasibility  of  the  air  defense  game  as  a  research  vehicle, 
volunteers  were  recruited  to  participate  in  a  preliminary  experiment.  The  expe-iment 
was  exploratory  and  was  designed  to: 


•  Illustrate  possible  research  uses  of  the  air  defense  game. 

•  Checkout  software  and  procedures  in  an  on-line  environment. 

•  Test  the  feasibility  of  extracting  detailed  measures  of  human  performance. 

•  Ascertain  the  amount  and  type  of  training  required  to  reach  stable  performance 
at  moderate  levels  of  difficulty. 

•  Derive  indices  of  performance  that  are  properly  sensitive  to  variations  in  task 

load. 

Subjects 

The  subjects  were  17  Navy  enlisted  men  (E-4  to  E-6)  waiting  to  enroll  in  advanced 
technical  courses  at  the  Naval  Training  Center,  San  Diego.  Their  median  age  was  19 
years;  the  range  was  from  17  to  28.  None  of  the  subjects  were  experienced  or  trained  in 
the  use  of  computers.  Participation  was  voluntary  and  informed  consent  was  solicited  in 
accordance  with  SECNAVINST  3900.39A. 

Procedure 


Two  subjects  were  run  at  a  time,  each  operating  a  separate  computer.  Neither 
subject  could  see  the  other's  keyboard  or  display.  The  testing  room  was  dimly  illuminated 
in  order  to  minimize  glare  on  the  display  screens. 


IS 


The  subjects  were  first  given  a  briefing  that  described  the  rationale  for  the 
experiment  and  gave  an  overview  of  the  air  defense  game.  This  was  followed  by  a 
detailed  introduction  to  the  game,  the  activities  required  during  an  engagement,  and  the 
type  of  feedback  to  the  player.  Participants  next  played  two  or  three  practice 
engagements  to  become  familiar  with  the  display  and  the  data-entry  keys.  There  were  six 
targets,  two  at  each  speed,  in  each  of  the  practice  runs.  The  experimenter  instructed  the 
subjects  in  interpreting  the  display  and  in  the  proper  use  of  the  function  keys.  The 
experimenter  emphasized  the  importance  of  avoiding  hits  on  the  ship  and  of  kil'ing  targets 
at  the  greatest  possible  range,  but  no  explicit  rules  were  given  as  to  when  to  fire.  Rather, 
these  were  to  be  learned  during  the  course  of  play. 

After  the  briefing,  practice,  and  instruction,  the  subjects  began  the  training  sequence 
proper.  They  were  required  to  master  each  of  four  blocks  of  progressively  more  difficult 
types  of  engagements.  Training  was  self-paced,  and  a  skill  rating  of  R^75  on  three 
consecutive  engagements  was  the  criterion  for  proficiency  and  the  prerequisite  for 
advancing  to  the  next  block.  Two  training  sequences  were  used  and  subjects  were 
assigned  randomly  to  one  or  the  other.  The  two  sequences,  denoted  as  A  and  B  in  Table  2, 
are  identical  in  blocks  1  and  4.  They  both  progress  from  an  easy  task  (18  targets  at  low 
tempo)  to  a  moderately  difficult  one  (36  targets  at  high  tempo). 


Table  2 


Number  of  Targets  and  Tempo  of  Operation  in  the  Two  Training  Sequences 


Training 

Sequence 


_ Number  of  Targets-Tempo  of  Operation _ 

Block  1  Block  2  Block  3  Block  4 


A  18-low  18-intermediate  36-intermediate  36-higl 

B  18-low  36-low  18-high  36-higl 


Tempo  was  deemed  to  be  more  important  than  the  number  of  targets  in  determining 
task  difficulty.  With  this  in  mind,  note  that  sequence  B  contains  an  abrupt  shift  in  tempo, 
between  blocks  2  and  3,  and  that  sequence  A  provides  two  blocks  of  training  in  the 
intermediate  tempo.  It  was  hypothesized  that  subjects  using  sequence  A  would  complete 
training  more  quickly. 

After  a  subject  reached  criterion  in  block  4,  ten  additional  engagements  were  run  in 
the  36  target-high  tempo  condition.  These  were  identical  to  those  of  block  4,  except 
that  there  was  a  programmed  30  second  rest  period  between  engagements.  At  the  end  of 
the  fifth  engagement,  an  auditory  monitoring  task  was  introduced.  The  subject  received 
instructions  and  six  minutes  of  practice  on  the  monitoring  problem  alone.  Then,  in 
engagements  6-10,  both  tasks  were  performed  concurrently.  The  monitoring  task 
continued  during  the  30-second  breaks  from  the  air  defense  game. 
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The  monitoring  task  resembled  a  typical  communications  problem  that  might 
accompany  AAW  operations  and  was  selected  because  of  its  minimal  sensorimotor  conflict 
with  the  air  defense  game.  Specifically,  the  subject  listened  to  a  series  of  words  and 
numbers  and  wrote  down  the  digits  that  followed  each  occurrence  of  a  preassigned  word, 
Ilach  message  in  the  series  consisted  of  a  category  (speed,  altitude,  etc.)  followed  by  a  1- 
word  call  sign  and  a  3-digit  random  number.  To  illustrate,  suppose  the  subject  had  been 
assigned  the  call  sign  YANKEE  and  had  been  presented  the  auditory  input  stream, 
"ALTITUnE  BRAVO  EIGHT-ZERO-SIX.  .  .SPEED  YANKEE  TWO-FOUR-THREE.  .  .  ." 
The  subject’s  task  was  to  write  "243"  on  a  response  sheet.  For  this  subject,  all  messages 
with  a  call  sign  other  than  YANKEE  were  to  be  ignored. 

The  series  of  messages  was  recorded  on  tape  and  played  through  Audiotronics  HS14 
headphones  at  a  rate  of  one  message  per  3.6  seconds.  Thus,  100  messages  would  be 
presented  in  6  minutes,  which  was  the  approximate  duration  of  an  engagement,  including 
the  rest  period.  Each  of  10  different  call  signs  appeared  at  10  random  times  in  every 
series.  A  given  subject  had  the  same  call  sign  throughout  all  sessions. 

Testing  was  scheduled  for  3  hours  on  each  of  2  consecutive  days.  At  the  end  of 
testing,  the  subjects  were  interviewed  for  their  evaluation  of  the  task,  and  their  questions 
about  the  experiment  were  answered. 

Results 


The  feasibility  of  the  software  concept  and  its  implementation  were  confirmed.  The 
on-line  collection  of  performance  data  proved  feasible  and  straightforward,  and  there  was 
no  disruption  to  the  user.  Minor  problems  were  found  in  the  interrupt  routine  that 
services  the  function  key  inputs,  and  these  were  corrected  in  the  final  version  of  the 
program. 

I3ue  to  an  error  in  programming  design,  the  inter-update  interval  in  the  experiment 
was  allowed  to  vary  from  7.0  seconds  in  the  18  target-low  tempo  condition  to  9.8  seconds 
in  the  36  target-high  tempo  condition.  The  resulting  duration  for  these  engagements 
ranged  from  4.2  to  4.9  minutes.  The  effect  of  this  variation  was  to  give  the  subjects  more 
time  per  update  in  the  more  difficult  engagements.  While  this  error  did  not  affect  our 
interpretation  of  the  results  that  follow,  it  constituted  an  undesirable  and  unintentional 
manipulation.  The  program  was  modified  to  equate  the  intervals  at  1 1.7  seconds  for  all  12 
combinations  of  the  tempo  and  number  of  targets.  This  modified  version  of  the  program 
appears  in  Appendix  A  and  is  the  one  recommended  to  potential  users. 

Skill  Acquisition 

Only  8  of  the  original  17  subjects  completed  the  training  regimen.  Eight  others  had 
conflicting  duty  assignments  and  one  failed,  after  30  training  engagements,  to  reach  the 
performance  criterion  for  Block  1;  data  for  these  subjects  have  been  excluded  from  the 
analysis  given  below.  Of  the  remaining  eight  subjects,  three  received  training  sequence  A 
and  five  received  sequence  B.  The  overall  mean  number  of  engagements  to  complete  the 
four  blocks  of  training  was  39.1  (median  =  40;  range  =  19  to  73).  The  mean  figure  is 
equivalent  to  2.9  hours  of  playing  time. 

The  number  of  engagements  required  to  complete  the  A  and  B  sequences  did  not 
differ  significantly  by  the  Wilcoxon  rank-sum  test;  the  medians  were  45  for  sequence  A 
and  35  for  sequence  B.  This  was  contrary  to  the  expectation  that  the  A  sequence  would 


provide  a  smoother  transition  in  difficulty  and  would  thereby  enhance  acquisition. 
Apparently,  the  rate  of  acquisition  was  insensitive  to  both  tempo  and  the  number  of 
targets.  The  Kruskal-Wallis  test  was  employed  to  examine  the  interaction  between  blocks 
and  training  sequence  (Bradley,  1968,  pp.  138-141).  The  interaction  was  nonsignificant, 
indicating  that  performance  for  the  four  component  blocks  did  not  reliably  differ  as  a 
function  of  the  training  sequence  manipulation. 

After  collapsing  the  nonsignificant  sequence  variable,  the  median  numbers  of 
engagements  to  criterion  were  9.0,  3.5,  5.5,  and  13.0  for  the  Blocks  1-4  respectively. 
These  differed  reliably  by  the  Friedman  test,  S(3)  =  153.5,  p  <  .01.  It  appears  that  once 
the  basic  skills  of  the  game  are  acquired  (in  the  easy  condition  of  Block  1),  then  these  are 
sufficient  to  prosecute  engagements  of  moderate  difficulty  (Blocks  2-3).  Additional  skills 
must  be  mastered,  however,  to  meet  the  increased  demands  of  the  36  targets-high  tempo 
condition  of  Block  4. 

Because  there  were  large  individual  differences  in  the  number  of  engagements 
required  to  reach  criterion,  performance  scores  were  Vincentized  (see  Hall,  1966,  p.  64) 
into  quartiles  to  further  examine  the  course  of  acquisition.  Figure  10  shows  the  mean 
skill  rating  (R)  by  quartile  in  each  of  the  four  training  blocks.  The  positive  trend  within 
each  block  makes  it  clear  that  performance  does  indeed  improve  over  the  four  quartiles. 
This  effect  was  significant  by  a  Friedman  test,  x^(3)  =  19.7,  p  <  .01.  Bradley's  (1968) 
suggestions  for  extending  the  Friedman  test  were  used  to  assess  possible  interactions; 
quartiles  did  not  significantly  interact  with  either  block,  x^(12)=8.33,  or  with  the 
training  sequence  (A  vs.  B),  =  2.0.  Thus,  the  course  of  acquisition  was  similar  within 

a  training  sequence  and  regardless  of  the  sequence  type. 

In  summary,  about  3  hours  with  either  training  regimen  produced  proficient  per¬ 
formance  in  quite  difficult  air  defense  problems.  The  course  of  skill  acquisition  was 
insensitive  to  the  variations  in  the  sequence  of  engagement  types.  Basic  skills  were 
acquired  largely  through  early  practice  and  transferred  readily  to  moderately  difficult 
engagements.  Significantly  more  practice  was  required  to  establish  a  high  level  of 
proficiency,  though  well  short  of  mastery,  in  the  more  demanding  air  defense  problems. 

Dual-task  Performance 


Only  five  subjects  were  available  to  complete  testing  in  the  dual-task  sessions.  The 
skill  ratings  declined  significantly  when  the  auditory  monitoring  task  was  added  to  the  air 
defense  problem.  Presumably,  the  information  processing  demands  of  the  two  concurrent 
tasks  exceeded  the  subjects'  resources.  The  differences  between  single-  and  dual-task 
performance  are  summarized  in  Table  3. 

Analysis  of  other  air  defense  game  data  corroborated  the  decrease  in  the  skill  rating 
observed  in  the  dual-task  condition.  The  range  in  miles  at  target  intercept  decreased 
while  the  number  of  hits  on  the  ship,  the  number  of  inflight  launches,  and  the  number  of 
splashes  increased. 

A  decline  in  performance  was  also  evident  in  the  auditory  monitoring  task.  When  it 
was  the  sole  task,  performance  was  virtually  perfect.  When  it  was  performed  concur¬ 
rently  with  the  air  defense  game,  detection  of  the  call  sign  messages  exhibited  a  small  but 
significant  decline.  Most  errors  (92.3%)  were  due  to  omissions.  From  this  pattern  of 
results,  it  is  clear  that  the  subjects  did  not  perform  one  task  strictly  at  the  expense  of  the 
other;  rather,  performance  in  both  tasks  was  significantly  affected. 


BLOCK  1  BLOCK  2  BLOCK  3  BLOCK  4 


1230  1234  1234  1234 

Quartile  Quartile  Quartile  Quartile 


Note.  Data  f  rom  training  sequences  A  and  B  were  combined. 

Figure  10.  Vincentized  mean  skill  ratings  (R)  by  quartile  for  the  four  training  blocks. 


Table  3 

Comparison  of  Performance  in  Single-  and  Dual-Task  Conditions 


Performance 

Characteristic 

Median  Scores 

Single-task  Dual-task 

Condition  Condition 

Skill  rating  (R) 

77.0 

68.0 

<  .01 

Target  intercept  (kill)  range 

15.8 

15.0 

<  .01 

Hits  on  ship 

0.0 

2.0 

<  .01 

Inflight  launches 

3.5 

6.7 

<  .025 

Premature  launches  (splashes) 

6.0 

5.0 

>  .10 

Monitoring  task  errors^ 

0.0 

1.0 

<  .01 

Wilcoxon  signed-rank  test. 

*^The  range  of  errors  was  0  to  1  when  the  monitoring  task  was  performed  by  itself;  the 
range  was  0-5  in  the  dual  task  condition. 
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The  relation  of  task  load  (single  vs.  dual)  and  target  speed  (fast,  medium,  or  slow)  to 
the  observed  firing  lag,  L-L*,  is  shown  in  Figure  11.  The  auditory  monitoring  task 
significantly  increased  firing  lags,  W  (5)  =  0,  p  <  .05,  Wilcoxon  signed-rank  test.  The  main 
effect  of  target  speed  was  also  significant,  S(3,5)  =  38,  p  <  .05  (Friedman  test).  Firing 
lags  were  longer  for  slow  targets  than  for  either  fast  or  medium  ones.  We  cite,  as  the 
most  likely  reason,  the  low  priority  that  the  rules  impose  on  slow  targets.  Since  they 
move  a  shorter  distance  between  updates,  smaller  penalties  are  incurred  (in  terms  of 
killing  range)  by  firing  at  fast  and  medium  speed  targets. 

The  interaction  between  target  speed  and  task  load  was  not  significant,  suggesting 
that  the  demands  of  the  monitoring  task  induced  a  general  decrease  in  attention  to  the  air 
defense  game. 


Note.  A  lag  of  2  means  that  the  player  fired  on  the  second  update  after  the  target 
entered  firing  range. 


Figure  11.  Mean  firing  lag  as  a  function  of  target  speed 
for  single-  and  dual-task  conditions. 
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Evaluation  by  Users 

Unlike  most  laboratory  tasks,  the  air  defense  game  was  fun  and  the  subjects  were 
able  to  maintain  their  interest  in  it  for  periods  of  3  to  4  hours  at  a  sitting.  They  reported 
that  the  feedback  displayed  at  the  end  of  each  engagement  was  especially  helpful  in 
diagnosing  their  tactical  weaknesses  and  in  providing  objective  performance  standards 
with  which  to  monitor  their  progress.  The  capability  for  presenting  engagements  of 
varied  difficulty  was  also  regarded  as  a  desirable  feature. 


CONCLUSIONS 

This  effort  demonstrates  that  the  air  defense  game  is  an  effective  vehicle  for 
investigating  human  performance  processes  in  AAW  threat  analysis.  The  game  requires 
sustained  attention  to  a  complex  and  interactive  "hostile"  environment,  provides  proper 
experimental  control  of  relevant  variables,  and  affords  detailed  quantitative  measure¬ 
ments  of  human  performance  that  can  be  compared  to  that  of  a  mathematically  ideal 
information  processor.  The  hardware/software  system  provides  flexibility  and  portability 
in  an  inherently  motivating  task  that  taps  the  cognitive  skills  required  in  selected 
command  and  control  environments. 

Proficient  performance  can  be  attained  with  moderate  amounts  of  practice.  Varia¬ 
tions  in  the  number  of  targets,  target  speed,  and  the  pace  of  operations  produce  reliable 
effects  on  performance.  The  task  is  sensitive  to  the  workload  demands  of  a  concurrent 
auditory  monitoring  task. 


RECOMMENDATIONS 

1.  The  air  defense  game  should  be  exploited  as  a  tool  in  future  research  on  the 
threat  analysis  problem. 

2.  The  air  defense  game  should  be  used  to  determine  limitations  in  human 
information  processing  and  how  strategies  for  threat  analysis  change  as  a  function  of  task 
load. 


3.  Performance  in  the  air  defense  simulation  should  be  used  as  a  dependent  measure 
for  other  military  research  applications  (e.g.,  the  effect  of  extended  effort  or  sleep  loss). 
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460  PRINT  "TED  FOR  EACH  NILE  THAT  YOUR  AVERAGE  KILL  RANGE  IS  UNDER  20." 
470  PRINT  ’i  TO  LAUNCH  A  NISSILE.  USE  THE  TEN  WHITE  KEYS  AT  THE"; 
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'iOO 

43d 

:.'CO 

0 

bro 
5  50 
540 
550 
560 

5  70 
560 
500 
600 

6  10 
600 
6  JO 
640 
650 
660 
670 
680 
600 
700 
710 
700 
730 
740 
750 
780 
770 


PRINT  ■■  UPPER  LEFT  06  THE  FEYBOARC  ENTER  ThE  TwO-DICJT  ■ , 

PRINT  -TRACA  NUHBER  OF  TmE  TARGET.  A  READOUT  WILL  THEN  INFORM  ' 

PRINT  YOU  OF  A  SUCCESSFUL  LAUNCH--OR  AN  ERROR 

PRINT  "AN  ERROR  OCCURS  IF  yqu  aEy  A  NUMBER  INCORRECTLY", 

PRINT  "  OR  IE  YOU  LAUNCH  A  MISSILE  UNNECESSARILY  " 

PRINT  "J  COCO  LUCA..  THE  Fate  of  YOUR  SHIP  lies  in  YOUR  HANDS'" 


PRINT  -j  Press  RETURN  to  cont-nue. 

input  as 

REM- - cleonup  memcrr  and  get  ployer  I  0. 


PRINT  'LJJENTEB  YOuR  NAME 
input  Si" 

01  -.0 
T7^54 
in  -  "L**  " 

Nit:'  “ 

delete  P3.S5.F3.C3,A3.L3.XO.A5.TS,A6.A7.A8.Ag,H9,L*,AI,A5,A4.w,70 
SET  nosey 
A=0 
TS  =  "  ■ 

REM - select  te»pc  onO  forget  oensity 

WS^'LOW 

.Y»="  INTERMEDIATE 

YS;"hICH 

Ot=' Proci . ce" 

PRINT  "JJSELECT  TEMPO.  U'.WS."  3=",YS 

PRINT  "  4=";0t,'  ' 

INPUT  0 

IF  D<1  OR  D>4  THEN  740 
IF  0<>4  Then  810 
N0  =  2 


780  n-2 
700  w7i443 
800  GO  TO  830 

810  PRINT  "JJJSELECT  target  DENSITY  II,  0,  3.  OR  4 1  :  " . 

820  input  N0 

830  IF  N0<1  OR  N0>4  then  820 
840  N0=N0*6 
850  A=N0/D 

862  REM - - set  W7  counter  to  eouote  updote  tines 

8  70  AS  =  SEG I "083095 1 000250220 180 1 000600200400 1000", 3»N0/2y3»D- 11,31 

883  WTsYAL (AS  1 

830  N-sS'NO 

900  W8=N=72  and  0=5 

912  PRINY  "JJJJWe'Il  Degin  ;n  o  fee  seconds 
920  CIM  A5INi?RI INI 

930  REM - random  I  zoi  1  on  for  entry  times 

940  call  "FLASH" 

950  R'  =  INT IRNOl M  =0  5) 

960  fcr  1=1  TO  A 

970  CO  TO  0  OF  1040,1010,980,1040 

950  A5  I  I  "OyM  =5»  I  mRI  1 1  ♦2»A  1 

900  45  ;  Im2»AiN01  =  4»  .'y3  =  RI  (  )»2»a->N0) 

1000  45ll72SAY2»N0)=I=Ri I :m2»Kt2SN0l 

1010  A5 I  I tAl =5sl *4yRI  I ImAl 

1020  A5  :  :i-A*N0i  =4s  !■»  I-R  1  I  I-.AYN0: 

1030  A5  (  I-tS-tOANOI  =|mRl  UmA»2»N0: 

: 040  A5 1! 1 =5SI»RI (1) 

1050  A5  ;  ItNOl  =4*1  -  I  «R1  (  I-.N01 
1050  A5;  I=2*N0I=2»I-IyRI  U=2*N0! 

■070  NEXT  I 

1080  REM - assign  trocK  nos.  speeos ,  ongles,  and  n.t  times 

I  090  DIM  P3190I , S3 (901 ,HO(Ni ,LS i I  200 1 , A1  190)  , A8 1901  . A9 1901  . X9 1901  , WINI 
I  100  S3  =  0 

1 1 10  call  ••mflash" 

I  120  AI=FNXI4) 

I  I  30  A8  =  FNXI  1 1 +9 
1  I  40  W  =  FNX I  I  1 
I  150  REM 

1 160  FOR  A=l  TO  90 
1170  B  =  A I  (A  I 
1  180  C  =  A8(M 
I  190  CALL  "MFLASH" 

1200  J  =  INTIRNDf1  l»a01«t 
1210  JI=INTIRN0(I )»O0)«I 
1220  REM 

1230  Al rA)=A) IJ) 
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1240  At (J)*8 
1250  A8(M=A0IJ1  I 
12G0  A8 IJ 1  I =C 
1270  NEXT  A 

t280  DIM  At  (M  .A8IN(  ,A4IN) 

1200  F08  1>'  10  N 
1  iZ0  AOI n :AI  I  I  1/4 
1510  IF  l<=NO  THEN  1390 
1320  IF  I<=2»N0  THEN  1390 
1350  A4(|;=28*A5in 
1  540  S3  I A9I I  1 1  =  1 . 6 
1350  CO  TO  1410 
1  360  A4(  I  1  =  lG«A5in 
1 370  S3(A9(I I  1-3 
1380  CO  10  1410 
1  3Se  A4  (  I  1  slO-'ASI  I  I 
1400  S3  I A9I I  11 =5 
1410  NEXT  I 

1420  8£Fi - draw  circles 

1430  PACE 

1440  cm  X  I  1  01  1  ,  Y  (  101  !  .T2(N) 

1450  20=46 
1460  COSC'8  1500 
1470  20=20 
1480  CCSUB  1500 
1490  CC  10  1570 
I  500  call  -nFLASH" 

15I0  X=SIN(FNX(3  611*20-77 
1520  Y=C0S(FNXf3  611*20-50 
1530  XI 1 1=-X 1  I  1 
1540  REM 

1550  call  -qflash-.x.y 

"SGO  RETURN 

1570  MOVE  77-0.5,48.5 

1580  PRINT  832, 185 

1590  PRINT  --t  “m* 

1600  DELETE  xTyTrT 

1610  REM-*-- - irlilolize 

1620  Om  *31901  ,C3I90I  ,A3nB01  .L31 1801  .TSI201  .A6IN1  .A7INI  .A5IN1 

1630  U=1 

1640  M=e 

1650  Ag=0 

1662  L7=e 

1670  L8=8 

1680  l9=0 

1690  A6=0 

1702  A7s0 

17  13  P3*=44 

1 722  FOR  1=1  TO  NO 

1 730  P3 I A9 I  I  1 1 =45 

1  743  NEXT  j 

I 75C  HO=0 

I  760  X9--0 

1772  149=1 

1780  Ll=1 

1793  SET  A£Y 

1800  A=l 

1810  REM - Stan  new  eooch 

1820  64x0 
1830  Hxt 

1840  call  •■flash- 
1850  A6=ABSIAS1«Hg 
1860  HBxl 

1870  REM - deiermine  large*  siotus 

1880  FOR  10=1  TO  N 

1890  CO  TO  2*1A5I101>U1-W8  OF  1900,2330.1980 

1900  IF  UxA4II0l  THEN  3420 

1910  X9t Agi 101  I x1 

1920  IF  P3rA9(I0n<44  THEN  I960 

1930  A4xA4-I 

1940  A5IA4I=I0 

1950  CO  TO  1980 

1960  CO  TO  A6liei-I  OF  1980.2570 

1970  461101=1 
1980  NEXT  10 

1990  REM - moke  rodor  eeeep 
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.?300  CHL  -nFLASH- 

7013  Fi  =  (?»X9'  I  1  »  ISINIFNX  I  41  )»P$.'t7l 
302C  CJ  =  C3S  IFNX  I  4  11  »P?-.50 
?0J3  Rtn 

3043  FUR  U 1  10  90 
,'300  CA^l  riF^ASP- 
3093  1  -3»  I  -  I 

3073  Ajijl  i-FJlll 
3383  A5  UI  .  I  1  =F3  (  I  1 
.■’393  L  ?  :  J  I  1  =C3  1 1  1 
3  I  03  L J  I  J  1  ♦ 1  l=CJI  1 1 
3  I  1  3  ‘(in 
3'3C  NEXT  r 
'  JO  call  -XElASM  ' 

3  I  43  IL}=  -A8S  lA  J1 
2150  n=-l 

3lS0  CALL  ■ OFLASH  '  AJ.L J 
3 ' TO  !F  44=0  THEN  3200 

2  I  80  REh - aisploy  lorge*  no8  . 

2:90  FOP  J2=l  TO  A4 
3300  C*.L  -nFLASH" 

2310  XT;SINIA) IA5I J21 1 1*48  6»’Tt0.5 

3230  tT;CCSIA1  1451  J21  1 1*48  5*48 

2350  REn 

2240  rOVE  XT . Y7 

2350  PRINT  'H'  a8  145  IJ2I 1 , 

2280  NEXT  J2' 

2270  REn - compuie  next  posiilcn 

2280  CALL  ■■FlAS‘4- 
22S0  P5=PJ-X9*S5 
2500  P5-.0  HAX  P5 
2510  u=U*l 
2520  CO  TO  1820 
2350  FOR  96=1  TO  W7 
2340  NEXT  wg 
2350  CO  TO  1980 

25S0  REn - ploi  ployer’s  m.sslle  irocX 

2570  A7 1101 =A7( 101*5 

2583  IF  A7iiai>20  Then  2500 

2  590  IF  P3  I  *9  I  lOI  1*S3  I  A9I  l0n-*7  I  101<  =  0  .  I  7HEN  2560 

2-lCO  C»ll  'nFLASH- 

2!I0  X  I  -SIM  A  I  I  101  )»A7  I  101  .17 

2420  >  ■ --COS  I  A  1  1 101  1*A7(  101  *50 

24^3  x3=S!NI*I II011*(A7I10!-|1*t7 

2443  t3:COS(*III0iI*IA7(I01-i; *50 

2450  REn 

24  60  nO>E  X  I  . r  1 

3470  DRA9  »2 , T2 

2483  CO  'C  1980 

2493  «En - "issile  sclosned 

2500  L8-.L8*I 
2510  ‘71101=0 
3520  A6UCi=0 
2530  CO  TO  1980 

2540  REn - Pesi'Oyed  o  icrgei 

2550  PRINT  'C  . 

3560  novE  77750 

2570  rotate  90-Al  I  101 

3580  RORAW  45.0 

2590  X91A9I [011 ,0 

2800  49=49. P3IA9( 10) )*S3IA9II0i 1 

2610  *51 I01=A5I 101 *2000 

2620  A6I 101 =0 

2630  41=41.1 

2640  IF  4I.H0ON  THEN  1980 
2S50  I0=N 
2660  NEXT  10 
2670  CO  TO  3560 

2680  R£n - - - ossendle  lorgei  no  For  Icunch 

2690  GO  TO  LENiTtl.l  OF  2700,3520.2730 
2700  T»=sec(T*. I .21 
27)0  C/N  xg, yg 
2720  GO  TO  3240 
2730  T0=valIT*I 
2740  CIN  Xg.yg 

2750  REn- 


loPAUp  torget  » 


?763  T?=T0=A8 

J-'70  T3=T?»W 

?'8a  j-SuniT2i 

?79c  if  j=P  then  J2I0 

aeKa  if  ABSi*6ijn  =  i  ihen  iisa 

2bl0  IF  \9lA9IJn=0  THEN  J2I0 
2a.?2  CALL  riFLASH- 
2r  JE  XJ-.SIMAI  (jn»5*T7 
284(5  Y3  =  C0S  I  A  I  I  J  I  1  »5*50 
.'853  x-(--SIMAI  IJI  )*4-T7 
2850  ir4^C0SIAI  I  J1  lA-l-SO 
2870  REn 

2830  pen - louncH  1  He  '"issile 

2830  HOVE  X3.T3 

2900  OPAW  XA,Y4 

2.910  A6!  Jl=-1 

2920  HS(J1=H 

2930  A7t j; =5 

2940  L3=l9«I 

2930  I»-STR(600»A8(J1  ) 

2960  COSUB  3340 

2970  REM - good  louncH  i  nf  crn>  ployec 

2980  FCR  S=l  TO  6 

2990  rOVE  I . 792*2 ) »5 . 376. 86. 912 

3000  PPIN"  T$: 

3010  FOR  SI =1  TO  8 
3020  NEXT  St 
3030  NEXT  S 

3040  IF  P3I A9IJ n *S3 I A9 I J) » >5  THEN  3300 

3050  REM - destroyed  target 

3060  PRINT  'C": 

3070  MOVE  T7.50 
3880  ROTATE  90-AI (J1 
3090  RCRAW  45.0 
3100  X9(A9IJI 1=0 
31 le  A6(JI=0 

3120  N9sNO*0  max  P3(A9(JI  )*S3(A9IJn 
3130  A5(J1=A51J)*2000 

3140  M  =M  *  1 

3153  IF  Nl*H0<>N  Then  3300 
3160  60  TO  3560 

3  170  REM - er ror  !  I n-<  1  I ght  lounch 

3183  L7^l7*I 

3190  1S=STRI700*A8IJ) I 

3200  C0SL.8  3340 

3210  REM . . error  .  i nf orto  ployer 

3223  CeSuS  3240 

3233  RETLRN 

3243  FOR  S=l  TO  6 

3253  MOVE  1  792*21*5  376.78  464 

3260  PRINT  T*. 

3270  for  Slil  TO  8 
3280  next  si 
3290  NEXT  S 
3300  MOVE  X6.V6 
3310  Tt=“- 
3320  RETURN 

3330  REM  load  L*  with  louncM  Informot ion 

3340  H»  =  STRILI*90  5-H/2I 
3350  h»=S£C(H*.2.3) 

3360  I*=SECf(«.2, 31 
3370  H»=H*SI« 

3380  L*=REPIH».LI ,01 
3390  LI=LI*6 
3400  RETUFiN 

3410  REM - torgei  hit  ployer'*  ship 

3420  H0=HB*l 
3430  A51I01=A5(t0)'*l000 
3440  XgiAgi 101  I =0 
3450  PRINT  'CCC"; 

3460  CALL  "MFLASH" 

3470  X8  =  SINIA1  (10)  1*48.5-»T7«0.5 

3480  Y8=C0S(AM  10)  )»48 .5*49 

3490  REM 

3600  MOVE  X8. 70 

3510  PRINT  "H**-; 
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5rre  ir  m.h2<>n  then  igse 
5sic  la-.N 
it-iu  NtXT  10 

3bb0  Pin - - - e^'d  of  gome 

rove  0.4 

3b’P  Pf?lNT  "End  of  game 
5‘:fl0  4--? 

ibqo  FCP  I = !  TO  700 
isac  ;--J 
3610  MIK’  I 
56. >0  Sl  ’  r.CfEf 

56  50  - - - sumnor  (  ze  ;^er f  cr irance 

5640  Z:n  C  I  n > . C2( j: . 7$  I5i 
3bbC  C'--0 
5S5C  ca-o 
5670  rcR  f-.-  ’0  3 
5660  fCR  ;  =  1  r,  -  I  UNO- I  70  4»N0 
5690  !r  451. 'I <0000  'hen  3700 
5  '00  00  :m  =C2  :4;  •  ■ 

5  ’  ■ 0  GO  TO  3730 
5  700  C!  IM;CI  ;m*i 
5’50  N-l^T  I 

3  7  40  \E)l1  4 

3750  :f  5'-0  Then  3770 
3760  '.9-49741 

3770  PRINT  "LIhhhh^haIR  DEFENSE  SI-Hhary 
3780  CO  1C  D'CE‘3790. 3910. 3830. 3640 
3790  Q»-.w» 

3800  CO  TO  3840 
3610  C«--X» 

3820  CO  TO  3840 
5850  Ct=Y» 

3840  PRINT  932. 18.0 

3850  PRINT  TErPO.  SPEED 

3660  V*="F4ST" 

3870  PRINT  USINC  3e20.V».C2M  )  .Cl  I  I  1 
3880  v$  =  "riEQluri' 

3890  PRINT  USINC  3920 . V« , C2 (2 1 . C 1 (21 
3900  V*="SLOW" 

39'0  PRIM  USING  3920  V*  .  C2  1 3  •  .  C  1  I  31 
39, '’C  IT-ACE  84.i0O.16D 
5930  PRINT  "J’OTAl  4!lLS  -s  '.SUNlCai.'  TOTAL  HMS  *  •■;SuniCl) 

3840  print  using  . JAVEfiACe  range  POR  4IlLS  =  •■20,20"  49 

3950  PRINT  '4l|;nA*  ■=  201" 

3E60  P»;  NC"6f  HISSIlES  " 

59'C  PRINT  R»,  launched  =  " . l9* L 7 . P$ , ' SPl ‘ShED  =  ".L8 
'3980  PRINT  "NO  OF  INFLIGHT  LAUNCHES  =  ";L7 
3590  IN'  I  IC0»4S.'20-  I2»SUh;c  1  )  -2»L7) 

4030  print  ;JS4lLL  RATING  =  "  .  R  “AX  0."  IITAX  =  1001" 

4010  IF  0*4  practice-  Then  4120 

4020  RETT - siore  0010  on  tope 

4050  PRINT  "Cold  is  oe t ng  s'oPeo  on  lope 

4040  ..=lENIL*1 

4053  find  FI 

4060  1484  I . 30»N*500‘L 

4070  find  FI 

4080  U--U-e  5-H/2 

4090  WRITE  933  S» . 0 . N . A  I . A5 , A8 . L . L* . C 1 . C2 . 43 . l9 . L8 , L7 , R . U 
4100  C.OSE 
4113  F I =F I ♦ I 

4120  PRINT  "JCO'TOU  WANT  TC  PL*T  AGAIN  lY/Nl’  " . 

4130  input  as 
4143  IF  A$i"Y"  THEN  4180 
4:50  IF  4$<>  N"  Then  4  I  30 
4 i 60  PRINT  "JThANK  you  ■ 

4  I  70  END 

4180  RETT - Se'  disploy  poromeiers  (on  ne»i  gome 

4190  PACE 
4200  A-2 
4210  T»I"" 

4220  SET  N04EY 
4230  n»='-  "in* 

4240  N»="  "4N« 

4250  2l=Zl4t 

4260  77-.T7*5  376 

4270  CO  TO  IZ1=5I«1  OF  630.590 


"  .SA.  ••  NO  OF  'AROE’S  =  .N 


»  4 Ills  »  hits 
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APPENDIX  B 

AIR  DEFENSE  GAME:  PROGRAM  VARIABLES 


VARIABLE 


NAME 

DEFINITION 

TYPE 

A 

Switch  for  servicing  function  keys 

Simple 

A$ 

Tape  file  header 

String 

A1 

Target  bearing(degrees) 

Array (N) 

A4 

Update  at  which  target  will  hit  own  ship 

Array (N) 

A5 

Target  entry  time:  plus  1000 (hit)  or  2000 
(killed) 

Array (N) 

A6 

Own  missile  status  (l^ln  flight) 

Array (N) 

A7 

Own  missile  position 

Array (N) 

A8 

Target  track  nuiid3er(TN) 

Array (N) 

A9 

Code  for  target  bearing»Al/4 

Array (N) 

B 

Temporary  storage:  bearing  randomization 

Simple 

C 

Temporary  storage:  TN  randomization 

Simple 

Cl 

No.  of  hits  (fast,  medium,  slow) 

Array (3) 

C2 

No.  of  kills  (fast,  medium,  slow) 

Array (3) 

D 

Tempo  of  operations 

Simple 

FI 

Magtape  data  file  no. 

Simple 

F3 

X-coordinate  for  target  display 

'viTay(9U) 

G3 

Y-coordlnate  for  target  display 

Array (90) 

H 

Missile  status 

Simple 

H$ 

Launch  data:  6  digits 

String 

HO 

No.  of  hits  on  own  ship 

Simple 

H9 

Missile  status 

Array (N) 

I 

Index  in  for-next  loop 

Simple 

1$ 

Launch  information 

String 

TO 

Index  in  for-next  loop 

Simple 

J 

Index  in  bearing  randomization 

Simple 

J1 

Index  in  TN  randomization 

Simple 

J2 

Index  in  for-next  loop 

Simple 

K 

Loop  control:  NO/D 

Simple 

K1 

No.  of  kills 

Simple 
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VARIABLE 

NAME 

DEFINITION 

TYPE 

K3 

Radar  sweep  array  for  FP-51 

Array (180) 

K4 

No.  of  targets  entering  this  update 

Simple 

K5 

Pointer  to  targets  that  enter  this  update 

Array (N) 

K9 

Accumulator  for  killing  range 

Simple 

L 

Final  length  of  L$ 

Simple 

L$ 

Player's  launch  history 

String 

LI 

Current  length  of  L$ 

Simple 

L3 

Radar  sweep  array  for  FP-51 

Array (180) 

L7 

No.  of  Inflight  launches 

Simple 

L8 

No.  of  splashes 

Simple 

L9 

No.  of  good  launches 

Simple 

M$ 

Display  of  launch  status 

String 

N 

Total  no.  of  targets 

Simple 

N$ 

Display  of  launch  status 

String 

NO 

No.  of  targets  of  each  speed 

Simple 

P3 

Target  range 

Array (N) 

Q$ 

Text 

Stri.'.y 

R 

Skill  rating 

Simp.ie 

R$ 

Text  for  feedback 

String 

R1 

O's  or  I's  for  entry  time  randomization 

Array (N) 

S 

Index  for  flashing  TN  display 

Simple 

s$ 

Player  I.D. 

String 

SI 

Index  for  flashing  TN  display 

Simple 

S3 

Target  speed 

Array (N) 

T$ 

Input  of  launch  data 

String 

TO 

2-digit  launch  data 

Simple 

T2 

Array  for  track  number  lookup 

Array (N) 

T7 

X-coordinate  for  own  ship 

Simple 

U 

Update  counter 

Simple 

v$ 

Text 

String 
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VARIABLE 

NAME 

DEFINITION 

TYPE 

W 

Track  number  lookup:  1,  2,...,N 

Array (N) 

w$ 

Text 

String 

W6 

Index  In  for-next  loop 

Simple 

W7 

Counter  to  equate  inter-update  intervals 

Simple 

W8 

Logical  switch  for  game  parameters 

1  =  high  tempo,  N  ■=  72;  0  =  otherwise 

Simple 

X 

X-coordinate  for  display  of  circle 

Array(lOl) 

x$ 

Text 

String 

XI 

X-coordinate  for  missile  track:  origin 

Simple 

X2 

X-coordinate  for  missile  track:  terminus 

Simple 

X3 

X-coordlnate  for  missile  track:  origin 

Simple 

X4 

X-coordinate  for  missile  track:  terminus 

Simple 

X7 

X-coordlnate  to  display  track  number 

Simple 

X8 

X-coordinate  to  overwrite  track  number 

Simple 

X9 

Target  status  for  display: 

0  =  Inactive;  1  =  active 

Array (90) 

Y 

Y-coordinate  for  display  of  circle 

Array (101) 

Y1 

Y-coordinate  for  missile  track:  origin 

Simple 

Y2 

Y-coordinate  for  missile  track:  terminus 

Simple 

Y3 

Y-coordinate  for  missile  track:  origin 

Simple 

Y4 

Y-coordinate  for  missile  track:  terminus 

Simple 

Y7 

Y-coordinate  to  display  track  number 

Simple 

Y8 

Y-coordinate  to  overwrite  track  number 

Simple 

Z 

Dummy  variable 

Simple 

ZO 

Radius  of  circle 

Simple 

Z1 

Counter  for  repositioning  display 

Simple 

B-3 
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APPENDIX  C 


DATA  ANALYSIS:  PROGRAM  LISTING 


C-0 


mm 


lea  Rfr. 

I le  R£n 
120  REM 
130  REn 
I  40  REM 
I5C  REn 
1E0  REn 
; 70  REM 
18C  INIT 
190  PRINT  "LENTER  PILE  NO.:  ; 

230  INPUT  Fi 
210  FIND  VAL  IF$1 
220  READ  S33  S*.O.N 

230  Din  A) INI .A5IN1 .A8fNl .0IN.5I .Cl (3) .C2I3) .WiaSI .YI3.4I 
240  READ  833: Al ,A5,A8.L 
2o0  Din  L*(L*I I 

2S0  read  e33.Lil.Ct  ,C2.A9,L9.Le.L7.SI  ,U 
270  KEfl  .  .  Oispioy  Ooia  Summery 

280  T»=“FILE  «"&F» 

290  PRINT  “LlAIR  DEFENSE  CAME  SUnMARY  • . T» , -  ".St 

300  CtiSECl'Cow  INTERMEOIATEHICH  ■■  ". 12»ID-II«I . 121 

310  PRINT  "JJNs  “.N;"  “;Ct:"  SPEED  IN  I LLS I  HI TSJ " 

320  FDR  Ul'TO  3 

330  Ql  =  SECr'FAST  MEDlunSLOW  "  . 6»  1 1- 11  ♦  I  . 6 1 
3A0  print  using  “eAlSDISO"  Ct.C2III.Cl  III 
350  NEXT  I 

360  PRINT  "JTQTAL  MILS  =  "  :SuniC21  ;  "  TOTAL  HITS  =  ";SuniCll 

370  print  using  JJAVE.  DISTANCE  FOR  FILLS  =  " "20 . 2D" i N9 

380  PRINT  "JeAuNChES's  ■■;L9*L7;"  SPLASHES  =  ".Le;"  INFLITES  s  ";L7 
380  PRINT  ■•■Rs  ".fil  MAX  0  " 

400  INPUT  Oi 

410  Din  RIU.Gl .F 131 .F4 131 ,F5(51 ,FBI3I ,F7I6I .HISUniCI 1  nAX  II 
420  READ  F,F4.F5.FG,F7 

430  DATA  2.5.12.9.15.28,5.3.1.6.46.44.44,1.2.4.5.10,20 
440  N0iN/3 
450  0:0 
460  v':l 


470  R:0 
480  N4*U 
490  X:0 

600  Y.J0 

510  S£n . WalooK-up  toble  for  lorgeTs  by  irocA  no 

520  REn . Oll.ll*Entry  U,  0 1 1 . 21  sOp  t  i  mol  U  for  lounen 

530  REn . 011.31=0  of  ployer’t  Killing  lounch 

540  REn . CI1,4I=»  of  inflignf  lounenes,  0(1.51=*  of  tplotnet 

550  REn  . Hsupdoies  on  which  hiiltl  occurred 


580  FOR  11=1  TO  3 

570  FOR  I=N0»(II-n*l  TO  N0»n 

500  W(A8(1  1  l:=I 

590  IF  A5  in  <2000  then  620 

600  01 1 . I  I =A51 11-2000 

610  CO  TO  650 

620  on.  1  l=A5l  1 1-  1000  - 

633  Hui=o(i,  ii=F4;n: 

643  J*JTI 

653  0II.2)=0(l,n+Flll1 
660  K4<K4  niN  0(1.21 
670  NEXT  1 
680  NEXT  II 


690  REn . Ce<  lounch  info  from  LtiTliUpdoie.  T2»TrocK  No. 

700  REn . Find  sploshes  ond  inflite  lounches 


710  FOR  1=1  TO  L/6 
720  XSiSECILS. 6*1-5. 31 
730  T1<VALIXS1-I00 
740  XStSECILt, 6*1-2. 31 
750  T2,vaL(X*1 
780  IF  T2>699  THEN  870 
770  XrT2-600 

780  IF  TloQIWIXI  .21  THEN  050 

790  REn . A  splash,  so  engoged 

800  orvfxi  .SliOftfIXI  .51*1 
810  FOR  J.Tl  TO  Tl*3 
820  RIJ,2I=RIJ,2I*I 
830  NEXT  J 
840  CO  TO  890 


AIR  DEFENSE  CAftE .  DATA  ANALYSIS 

R.  L.  Hershnon.  F.  L.  Crci»*er,  4  R.  T.  Kelly 
Command  and  Support  Sysiems 
Novy  Personnel  Peseorch  4  Oevelopneni  Center 
Code  302 

Son  Diego.  CA  92152 


C-1 


850  OIWIXI .3)iT| 

660  CO  TO  890 
870  XiT?-700 

880  0IWIXI.4)«QIWIX).4l*t 
890  NEXT  I 

600  REM . Prim  doio  by  torgai 

gie  FOR  TO  3 
920  PRINT  "L-iT*,- 

030  PRINT  -ANC  TN  E  L»  L  LAC  IN  SP 

940  FOR  JiN0«(l-ll»l  TO  I»N0 

050  Q6»QIJ. 3)-Q( J.2I  NAX  0 

060  vi[  ,  1 1  sY  ( J .  n••0(  J.51 

970  VI ] .21 xY II .2) 4Q6 

983  Y| • .31 *v  M . 31 406t2 

990  V  I  I  .  41 xY  1 1 . 41 tQIJ . 4) 

1000  At^SECC  FnS', 1 . 1 1  • 

1010  C»  =  SEC(  'HK“.0(J.31  niN  2.1) 

1020  PRINT  USINC  1040.  J,  A«.aI  ( J) ,  A8(.')  ,01 J  .  11  .0IJ.2)  .01J,31 
1030  PRINT  USINC  1 050 . 0 ( J . 31 -0 U . 21 , 0 1 J . 4 1 . C IJ . 5 1 . C« 

1040  iriACE  2C4XIA4X304X204X304X3D4X304XS 
•050  image  304X204X204XrA 
1060  NEXT  J 
1070  input  0« 

1380  NEXT  1 

1390  PRINT  ••  IHMHHMEAN  LAC  SO  LAC  N  IN  SP" 

I  100  FOR  l=i"T6'|' 

1113  C«=SEGr  FASTMEO  SLOW"  .4»U-1  1+1  ,41 

1120  PRINT  C*;  ■  TARGETS 

1130  PRINT  USINC  "2D.2D7XS".TI1.2I/C2II) 

1140  PRINT  USING  "20.207XS"  SOR ( I Y 1 1 , 31 -V  1 1 , 2 1 T2/C2 I  1 1 1 /C2 I  I  I  1 
1150  PRINT  USINC  "207X207X20“.C2m,YII,41,T(l,  11 
1  I  60  NEXT  I 

1170  X=YI I ,21«YI2,2I4Y(3.2) 

1180  SsY  1 1 ,31 ♦YI2,31«YI3, 3) 

1190  PRINT  USING  .  ALL  TARGETS  "”20 . 2D7XS " . X/SUMICO) 

1200  PRINT  USINC  ■■20T2D7XS"  .SORI  IS-Xt2/SUMIC2l  l/SuniC2)  1 
1210  PRINT  USING  "2D7X2DS",SUniC2) ,Y(1 ,4l4YI2,4l4YI3,4) 

1220  PRINT  USINC  "TXODv.YIl , ll♦Y(2, 1 1«YI3. 1 1 

1230  INPUT  0* 


240  REM . Aar  lys  is  oy  updot  e  .  .  .  R I J  ,  1 1  sAci  I  v« 

250  REM . Rt  J  ,21'«Engoged  RIJ.3)»AilIS,.  RIJ,4)sHils 

260  REM  . R(J,5l3UntrgQged. .  R ( J , 61 sM i seed  opps . 


1270  FOR  list  TO  3 

1280  FOR  IxNOTII I -I l4l  TO  NO* 1 1 

1200  A5»0I!.3I-1 

1300  IF  on, 31*0  then  1550 

1310  REM  . Process  ihe  Lill 

I  320  XIxFSI  1 1 1 -F51 1 1 1*101 J ,31-011 ,  I )I 

1330  X2*5 

I  340  AxQl  1 ,31-1 

1350  IF  XI<xX2  then  1410 

I  360  X  lxXI -F5(1 1 1 

1370  X2xX2«5 

1380  A*A4l 

1393  CO  TO  1350 

1400  REM . forget  is  octivc 

1410  FOR  J*0l I ,11  TO  A 
1420  RIJ, 1 IxRIJ, 1 )♦! 

1430  NEXT  J 
1443  for  JxA4l  TO  U 
1450  RIJ. 3lxR(J, 31*1 
1460  NEXT  J 

14  70  REM.  .  . forget  is  engoged 

1480  IF  A*IxOII,3I  then  1520 
1490  FOR  J*0I1,31  TO  A 
1530  RIJ. 2I*R(J. 21*1 
1510  NEXT  J 

1520  IF  011,31*011.21  then  1690 
1530  CO  TO  1620 

1540  REM . Process  the  hit 

1550  FOR  3*0(1,11  TO  0(1. I1*F4(II)-1 
1560  RIJ. 1 1>RIJ, 1 1*1 
1570  NEXT  J 

1580  FOR  J>OI 1 . 1 lvF4l 1 1 1  TO  U 
1590  RIJ. 4I*RIJ, 41*1 
1600  NEXT  J 


lEie  K 
1670  J 
1630  FI 
1640  f 
161:0  H 
1660  F 
1670  J 
1680  t 
1690  ^ 
1700  1 
1710  F 
I  770  t 
I  7  JO  1 
1740  I 
1750  1 
1780  I 
1  770 
I  780  : 
I  790  1 
1800 
1810 
1820 
1830 
1840 
I8S0 
I860 
1870 
1880 
1890 
1900 
1910 
1920 
1930 
1940 
19S0 
1960 
1970 
1  980 
1900 
2000 
2010 
202C 
2030 
2040 
2060 
2060 
2070 
2080 
2090 
2100 
21  10 
2120 
2130 

2140 

2150 

2160 

2170 

2180 

2190 

2200 

2210 

2220 

2230 

2240 

2250 


45iOI  I .  1 1*F4 ( 1 11-1 

PEM  . Unpngoged  torg«i*  ond  missdd  oppd ■ 

F0F»  J.bl  1 .2'  TO  65 
BIJ.G1=RIJ.61*JI 
RIJ.5l5F»(J.5l»l 
JliJM! 

NEKT  J 
next  I 

PR^Nt'-'l-.T..-,  U  act  eng  a  H  REtl  UT  M0.“ 
n-.a 

01=u- (RIU-I .31*R1U-I ,41=N) 

FOR  Isl  TO  U1 

PBiNT'’iiiNc'lUi!l:R‘ll:ii.R<I.2).R<I.J),RII.41.R5.RII.61.RtI,Sl 
image  303X2D3X203X2DSX2C3X2D3X2D343D 
MsM  MAX  Rli.S) 

NEXT  I 


INPUT  0*  ^ 

PEM . Moke  the  grppM .  .  oxe» .  t  i cks .  ond  vOlu«« 

PACE 

MOVE  10.10 
DRAW  110.10 
MOVE  10,10 
DRAW  10. 100 
X1:I00/UI 
X2=85/M 
FOR  1:1  TO  Ul 
MOVE  1»XI«10.10 
RORAW  0.-2 

IF  1/50INT  ( 1/51  Then  1940 

PRINT  USING  . HJ““2D*.I 

NEXT  I 

FOR  J=t  TO  6 

IF  INTIM/ISI  MAX  l<F7IJ»  THEN  1980 
NEXT  J 

FOR  1:0  TO  M  STFP  F7(J-I1 
MOVE  I0.I*X2715 

^F**t/ I5»F7  ( J- 1 )  1  <>INT  1 1/ I5»F7  ( J- 1 1 1 1  THEN  2040 
RMOVE  e.-l 

PRINT  USING  ••“••HHH”“30"  I J 
NEXT  I 
MOVE  0.78 

PRINT  -P  E.R.F.O.R.M.a.N.C.E”; 

MOVE  10.0 

PRINT  T$i"I  UPDATE-; 

MOVE  A4»XI»9.5,  1 1 
print  "Tf 

IF  SUM(CTI:0  then  2170 
FOR  1:1  TO  S'JMICI  1 
MOVE  H(I 1»XI«0.5,  1  I 
print 

p|J5^  * . Ploi  unengoged  torgei*  end  nl»»«d  cpponunl  i  i«i 

MOVE  XI79.RI I .GI4X2+I5 
FOR  1:2  TO  UI 
DRAW  I»Xl7S,RII,GI»X27l5 
NEXT  I 

FOR  1:1  TO  Ul 
;  MOVE  I»X1-*9,RU  .6)»X27I4 
!  PRINT 
I  NEXT  I 
!  END 
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APPENDIX  D 

DATA  ANALYSIS:  PROGRAM  VARIABLES 


D-0 


VARIABLE 


NAME 

DEFINITION 

TYPE 

A$ 

Target  speed  deslgnator(F,  M,  or  S) 

String 

A1 

Target  bearing(degrees) 

Array (N) 

A5 

Target  entry  time:  plus  1000 (hit) 
or  2000 (killed) 

Array(N) 

A8 

Target  track  number (TN) 

Array (N) 

c$ 

Outcome  designator (hit  or  kill) 

String 

Cl 

No.  of  hits  (fast,  medium,  slow) 

Array (3) 

C2 

Mo.  of  kills  (fast,  medium,  slow) 

Array (3) 

D 

Code  for  tempo  of  operations 

Simple 

F 

No.  of  updates  until  optimal  launch 

Array ( 3) 

F$ 

Input  for  file  to  be  analyzed 

String 

F4 

No.  of  updates  until  hit  occurs 

Array (3) 

F5 

Target  speeds 

Array (3) 

F6 

Range  at  entry  time 

Array (3) 

F7 

Scale  for  ordinate  in  MO  graph 

Array (6) 

H 

Updates  on  which  hits  occurred 

Array 

I 

Index  in  for-next  loop 

Simple 

11 

Index  in  for-next  loop 

Simple 

J 

Index  in  fcr-next  loop 

Simp  '.e 

J1 

Counter  for  missed  opportunities 

Simple 

K 

Update  at  which  kill  occurs 

Simple 

K4 

Update  for  first  launch  opportunity 

Simple 

KS 

Last  update  prior  to  a  hit 

Simple 

K9 

Average  distance  for  kills 

Simple 

L 

Length  of  L$ 

Simple 

L$ 

Player's  launch  history 

String 

L7 

No.  of  inflight  launches 

Simple 

L8 

No.  of  splashes 

Simple 

L9 

No.  of  good  launches 

Simple 

M 

Maximum  value  of  (R)  by  update 

Simple 

N 

Total  no.  of  targets 

Simple 

D-1 


VARIABLE 

HAME 


DEFINITION 


TYPE 


NO 

No.  of  targets  of  each  speed 

Simple 

Q 

Table  of  target  and  launch  Information 

Array(N,5) 

Q$ 

Text  for  game  parameters 

String 

Q6 

Lag  score 

Simple 

R 

Table  of  target  status  by  update 

Array(U,6) 

R1 

Skill  rating 

Simple 

R5 

No.  of  targets  remaining 

Simple 

S 

Sum  of  squares  for  lags 

Simple 

s$ 

Player  I.  D. 

String 

T$ 

File  number  designator 

String 

T1 

Code  for  update  extracted  from  L$ 

Simple 

T2 

Code  for  track  number  extracted  from  L$ 

Simple 

U1 

Last  update  of  game 

Simple 

V 

Table  for  track  number  lookup 

Array (9 9) 

X 

Sum  of  lags 

Slmpl e 

x$ 

Launch  Information  extracted  from  L$ 

String 

XI 

Temporary  storage  for  kill  computation; 
Unit  for  abscissa  In  MO  graph 

Simple 

X2 

Temporary  storage  for  kill  computation; 
Unit  for  ordinate  In  MO  graph 

Simple 

Y 

Summary  table  of  lags.  Inflight  launches, 
and  splashes  by  target  speed 

Array (3 ,4) 
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